Improving List Experiments

 

Gustavo Diaz
McMaster University
gustavodiaz.org

 

Slides: talks.gustavodiaz.org/tec

Research Agenda

Bias-variance tradeoff as darts

But the game of darts is more complicated

Two types of tradeoffs

  1. Explicit: Is a little bias worth the increase in precision?

  2. Implicit: Improve precision without sacrificing unbiasedness?

Two types of tradeoffs

  1. Explicit: Is a little bias worth the increase in precision?

  2. Implicit: Improve precision without sacrificing unbiasedness?

List Experiments

Example

List experiment

Here is a list of things that some people have done.

List experiment

Please listen to them and then tell me HOW MANY of them you have done in the past two years.

List experiment

Do not tell me which ones. Just tell me HOW MANY:

 

Control group

  1. Discussed politics with family or friends
  2. Cast a ballot for governor Phil Bryant
  3. Paid dues to a union
  4. Given money to a Tea Party candidate

List experiment

Do not tell me which ones. Just tell me HOW MANY:

 

Treatment group

  1. Discussed politics with family or friends
  2. Cast a ballot for governor Phil Bryant
  3. Paid dues to a union
  4. Given money to a Tea Party candidate
  5. Voted “YES” on the Personhood Initiative

Prevalence rate

\[ \text{Proportion(Voted yes)} =\\ \text{Mean(List with sensitive item)} -\\ \text{Mean(List without sensitive item)} \]

  • But we do not know how individual respondents voted!

Compare with direct question

Did you vote YES or NO on the Personhood Initiative, which appeared on the November 2011 Mississippi General Election Ballot?

\[ \text{Proportion(Voted yes)} =\\ \text{Mean(Voted yes)} \]

Validation

Validation

Validation

Validation